1. The Field of the Present Disclosure
The present disclosure relates generally to generating images for display, and more particularly, but not necessarily entirely, to a method and system for generating 3-D images for display.
2. Description of Related Art
Stereopsis is the visual ability to perceive the world in three dimensions (3-D). Stereopsis in humans is primarily achieved by the horizontal offset, known as interocular offset, between the two eyes. Interocular offset leads to two slightly different projections of the world onto the retinas of the two eyes. The human mind perceives the viewed object in 3-D from the two slightly different projections projected onto the two retinas.
One of the main ways in which human eyes perceive distance is called parallax. Parallax is an apparent displacement or difference in the apparent position of an object viewed along two different lines of sight. Nearby objects have a larger parallax than more distant objects when observed from different positions, so parallax can be used to determine distances. In humans, the two eyes have overlapping visual fields that use parallax to gain depth perception, that is, each eye views the object along a different line of sight. The brain exploits the parallax due to the different view from each eye to gain depth perception and estimate distances to objects.
This same method of parallax is used to give the illusion of distance in 3-D stereo images, including still images, videos, movies, whether captured by camera or computer generated. 3-D stereo images simulate real-world perception by displaying a slightly different image for each eye—a slightly different perspective of the same scene—where the viewing position is offset slightly in the horizontal direction (interocular distance). The two images that are displayed independently to the right and left eyes are sometime referred to as a “stereo pair.”
There are many methods for displaying a different image to each eye to generate the perception of a 3-D image. For still images, display methods may include a lenticular display surface, or a special viewing devices. For movies and videos, the display method may involve the viewer wearing glasses which permit a different color space or polarization to reach each eye, or which shutter alternating frames between right-eye views and left-eye views.
The perceived depth of an object may be determined by the angle at which the viewer's eyes converge. This is also the case when viewing a 3-D image that is displayed on a surface. Where both eyes view the same object in the same location, the object will appear to be positioned at the same distance of the display surface. This is because the eyes are converged at that distance just as they would be if an actual object were placed at that distance. When there is no separation between the images for the left eye and the right eye, this is referred to as zero parallax.
If the position of an object in the left eye's view is located to the right, and the position of the object in the right eye's view is located to the left, this is called negative parallax, and the eyes have to rotate inward (cross-eyed) to converge the images into a single image. In this case, the object is perceived to be located in front of the display surface.
If the position of an object in the left eye's view is located to the left, and the position of the object in the right eye's view is located to the right, this is called positive parallax, and the eyes have to rotate outward (more wall-eyed) to converge the images into a single image. In this case, the object is perceived to be located beyond the display surface. In short, when viewed in stereo pairs, an object must have negative parallax to appear closer than the display surface, and an object must have positive parallax to appear further away than the display surface. An object with zero parallax will appear to be at the distance of the display surface. Referring now to FIGS. 1A, 1B and 2, there are shown examples of how parallax allows a human to perceive distance.
In FIG. 1A, a distant object 10 is perceived by a human as a single image from the two images viewed by the left and right eyes. The index finger 12 is seen as a double image while viewing the distant object 10. In particular, the left eye sees the index finger 12 offset to the right by a distance 16 and the right eye sees the index finger offset to the left by a distance 14. In this case, there is a relatively small negative parallax.
In FIG. 1B, an index finger 20 is perceived by a human as a single image from the two images viewed by the left and right eyes. A distant object 22 is seen as a double image while viewing the index finger 20. The left eye sees the distant object 22 offset by a distance 24 to the left of the index finger 20 while the right eye sees the distant object 22 offset by a distance 26 to the right of the index finger 20.
Referring now to FIG. 2, there is shown an example of parallax and perceived distance of an image shown on a display surface 30. For purposes of this example, a triangle 32, a circle 34, and a square 36 are shown in the perceived locations, i.e., the locations where they are perceived to be located by the human mind. In regard to the triangle 32, both the left eye and the right eye see the single image of the triangle 32 at the same location. In the case of the triangle 32, there is zero parallax as the eyes converge to see the single image at the distance of the display surface 30, so the location of the triangle is perceived to be at the distance of the display surface 30.
In regard to the circle 34, the perceived location of the circle 34 is created by images 34A and 34B on the display surface 30. In particular, the left eye views the image 34A and the right eye views the image 34B such that the location of the circle 34 is perceived in front of the display surface 30. In this case, the eyes rotate inward (cross-eyed) to converge the images 34A and 34B into a single image, which is defined as negative parallax.
In regard to the square 36, the left-eye views the image 36A on the display surface 30 and the right eye views the image 36B on the display surface 30 such that the location of the square 36 is perceived beyond the display surface 30. In this case, the eyes rotate outward to converge the images 36A and 36B into a single image so that the square 36 appears to be further away than the display surface 30, which is defined as positive parallax.
3-D images of real-world objects may be captured by using two cameras, one for capturing the right-eye image and one for capturing the left-eye image. If a scene is to be viewed on a flat display surface in front of the viewer (such as on a television or movie screen), positive parallax can be captured by aiming the two cameras slightly toward each other (with a slight toe-in). The two cameras would both aim at a point along a central viewing axis. Optimally, this point would be the same distance from the cameras as the display surface will be from the audience. This way, the scene will appear correctly when viewed in 3-D stereo, with close objects having a negative parallax, objects at the distance of the display surface having zero parallax, and distant objects having positive parallax. These concepts are depicted in FIG. 3 as will now be explained.
FIG. 3 depicts a top view of a 3-D scene when captured for display on a flat display surface 50. The 3-D scene may be filmed using a left-eye camera 58 and a right-eye camera 60. The left-eye camera 58 and the right-eye camera 60 may be offset from a centerline, or y axis by an amount c, representing the interocular distance needed to create a 3-D image. There is shown a desired perceived position of a triangle 52, a circle 54, and a square 56 from the perspective of the viewer and in relation to the display surface 50. As used herein, the term “perceived position” may refer to the position where the viewer perceives the image in 3D.
In order to have the perceived position of the triangle 52 to appear at the same distance as the display surface 50, the triangle 52 is positioned along the y axis at the same distance as the display surface 50 and the aim of a left-eye camera 58 and the aim of a right-eye camera 60 converge at a distance equal to the distance of the display surface 50 from the viewer.
Circle 54 will appear to be located in front of display surface 50 because it is offset to the right in the left-eye camera view and offset to the left in the right-eye camera view (defined as negative parallax). Square 56 will appear to be located beyond the display surface 50 because it is offset to the left in the left-eye camera view and offset to the right in the right-eye camera view (defined as positive parallax).
Converging the aim of the cameras as described above will create positive parallax only in the direction of the camera convergence. In FIG. 3, for example, the cameras converge along the y axis. This will produce positive parallax in the direction of the y axis and enable display of 3-D objects that appear to be located beyond a flat display surface. This method works for flat display surface 50, because the position of display surface 50 is offset from the cameras in the direction of the y axis. But the aforementioned method is not suitable for images that will be viewed on a dome surface, as explained in the paragraph below.
In a dome environment, images are projected onto the inside of a hemispherical, dome display surface. These images may be captured with a dome camera which yields a 180-degree view, for example an astronomy image of the entire night sky. This dome camera may consist of a single camera with a circular fisheye lens, or a set of cameras, the images of which are assembled to create a hemispherical image. This dome camera may be a virtual camera or a real-world camera. The majority of the image is viewed high in the dome above, behind, and to the sides of the viewer, rather than just in front of the viewer of a flat surface display. To capture 3-D objects that will appear to be located beyond a dome display surface, right-eye and left-eye dome cameras capturing the scene must be aimed in a direction parallel to the central viewing axis, or undesirable effects will be produced: If the two dome cameras were aimed with a toe-in as described above for a flat display surface, positive parallax would only be produced in an area of the dome in the direction of the y axis directly in front of the viewer. As the viewer looks upward in the dome at angles above the y axis, the positive parallax effect diminishes and then reverses in areas of the scene overhead and behind the viewer. For example, consider FIG. 3 in three dimensions. If circle 54 were raised a great distance above the plane of the drawing (in the z axis direction, which would be above the viewer), positive parallax could never be achieved by the depicted camera convergence. Even at a great distance, circle 54 will always appear to be located in front of the dome display surface because it will be offset to the right in the left-eye camera view and offset to the left in the right-eye camera view (negative parallax). Positive parallax can only be created in the direction of camera convergence, in this case the y axis, and regardless of actual distance, the effect will actually be reversed on any object whose y component of distance is located in front of the point of camera convergence. In addition, if the dome cameras are angled to create the convergence on the y axis in front of the viewer, then objects overhead would be captured at different angles in the right-eye camera and the left-eye camera, so these overhead objects in the resulting image would appear to crisscross. The minds of the viewers would not be able to make sense of these anomalies, and the illusion of 3-D would be destroyed.
Therefore, cameras capturing 3-D stereo to be rendered on a dome surface must be parallel to each other (parallel to the central viewing axis). As a result, positive parallax cannot be captured from the original scene. So objects in the stereo images can only appear to be at located at the distance of the dome surface or closer to the viewer, and none will appear to be located beyond the dome surface.
The prior art is thus characterized by several disadvantages that are addressed by the present disclosure. The present disclosure minimizes, and in some aspects eliminates, the above-mentioned failures, and other problems, by utilizing the methods and structural features described herein. The method described in the present disclosure allows positive parallax to be captured in front of the viewer, upward in an arc through the zenith of the dome, and beyond to the back of the dome. The method allows scenes to contain positive parallax on a dome surface (and therefore allows objects to appear to be located beyond the dome surface when viewed in 3-D stereo), which was previously not possible.
The features and advantages of the present disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by the practice of the present disclosure without undue experimentation. The features and advantages of the present disclosure may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims.